{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 生成器介绍"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 基础知识"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "生成器是python的一个特别特的特性，在许多场合都有重要应用。比如range函数产生的就是一个生成器。其主要的好处就是降低了内存的占用。为什么呢？拿range函数来讲吧，它的目的是生成一系列的数。假如我们想生成一列数0，1，2，3，4，则"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "range(0, 5)"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "range(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "欸？怎么回事，输出的不是0，1，2，3，4,而是一个函数。这个函数实际上便是一个生成器。我们可以采用下面的方法把它展开"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[0, 1, 2, 3, 4]"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "list(range(5))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "或者使用一个for循环"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0\n",
      "1\n",
      "2\n",
      "3\n",
      "4\n"
     ]
    }
   ],
   "source": [
    "for i in range(5):\n",
    "    print(i)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "生成器常常与for循环结合使用。在上述循环过程中，每循环一步，range产生一个数，直至range抛出一个结束的异常，for捕获异常后结束循环。\n",
    "通过运行步骤，我们可以明白生成器节省空间的道理所在。range并没有一次性将数列都生成，而是逐渐生成。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 自制生成器\n",
    "为了进一步掌握生成器的编写方式，我们一起来实现range函数。当然这不一定是python内部的实现。如前文所述，range函数的作用是生成一列数。range的原本功能可以指定起始数、终止数和间隔。为了演示突出生成器的重点，我们将实现一个简化版的range，只给一个输入n，让其生成由0到n-1的数。并且，这个函数只是我们自己使用，就不做输入合法性检查了。闲言少叙，先看代码"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0, 1, 2, 3, 4]\n"
     ]
    }
   ],
   "source": [
    "def _range(n):\n",
    "    step = 0\n",
    "    while step < n:\n",
    "        yield step\n",
    "        step = step + 1\n",
    "print(list(_range(5)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "注释讲解版函数如下"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 为了与python内置range函数进行区分，我们将其命名为_range.\n",
    "# 那么如果你说我就不区分会怎么样？如果不区分的话，那么咱在这里定义的函数便会覆盖原函数。\n",
    "def _range(n):\n",
    "    # 采用一个变量暂存步数\n",
    "    step = 0\n",
    "    # 采用一个while循环，产生数据。\n",
    "    # 当步数小于输入值n时，执行循环迭代\n",
    "    while step < n:\n",
    "        # 这是迭代器最关键的一步。yield表示输出。\n",
    "        # 每当调用一次_range，便会执行到一个yield处，产生一个数据，同时暂停函数运行，直至再次调用。\n",
    "        # 稍后会给出另外一个版本，加强大家在这方面的理解。\n",
    "        yield step\n",
    "        # 当再次调用时，会从这里开始，而非函数头部。步数加1，继续迭代\n",
    "        step = step + 1\n",
    "# 调用并将产生的数据展开后输出\n",
    "print(list(_range(5)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "为了进一步加强大家对生成器的理解，接下来我们将上述版本稍作修改。我们想要产生一个0，0，1，1，2，2.。。。n-1,n-1的数列。该如何是好？且看"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0, 0, 1, 1, 2, 2, 3, 3, 4, 4]\n"
     ]
    }
   ],
   "source": [
    "def _range(n):\n",
    "    step = 0\n",
    "    while step < n:\n",
    "        yield step\n",
    "        yield step\n",
    "        step = step + 1\n",
    "print(list(_range(5)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "同时采用了两个yield，每当对_range函数调用一次，便从函数暂停的位置运行至下一个yield处，产生数据，随后输出。\n",
    "接下来再举一个例子。我们产生一个0，0，1，0，1，2，0，1，2，3等等。这就要用到yield from这个语法了。它是生成器中的生成器。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0, 0, 1, 0, 1, 2, 0, 1, 2, 3]\n"
     ]
    }
   ],
   "source": [
    "def _range(n):\n",
    "    step = 0\n",
    "    while step < n:\n",
    "        # range是另外一个生成器，yield from表示函数运行到此处时，进入range函数继续执行，直至range函数执行完毕。\n",
    "        yield from range(step)\n",
    "        step = step + 1\n",
    "print(list(_range(5)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "如果不用yield from会怎么样？想想。答案如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[range(0, 0), range(0, 1), range(0, 2), range(0, 3), range(0, 4)]\n",
      "range(0, 0)\n",
      "range(0, 1)\n",
      "range(0, 2)\n",
      "range(0, 3)\n",
      "range(0, 4)\n",
      "[]\n",
      "[0]\n",
      "[0, 1]\n",
      "[0, 1, 2]\n",
      "[0, 1, 2, 3]\n"
     ]
    }
   ],
   "source": [
    "def _range(n):\n",
    "    step = 0\n",
    "    while step < n:\n",
    "        # range是另外一个生成器，yield from表示函数运行到此处时，进入range函数继续执行，直至range函数执行完毕。\n",
    "        yield range(step)\n",
    "        step = step + 1\n",
    "print(list(_range(5)))\n",
    "for i in _range(5):\n",
    "    print(i)\n",
    "for i in _range(5):\n",
    "    print(list(i))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 一个实际应用\n",
    "不知道大家对生成器的基本原理及用法是否理解？下面我用一个更为实际的例子讲解生成器的妙用。\n",
    "我在做数据处理时，单个数据文件可能非常大，直接读进内存很困难。这时候，便可采用生成器逐步读取处理。如"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "def getFrame(filename):\n",
    "    pos = 0\n",
    "    frameSize = 100\n",
    "    fullSize = os.path.getsize(filename)\n",
    "    with open(filename, 'rb') as fid:\n",
    "        while pos < fullSize:\n",
    "            fid.seek(pos)\n",
    "            yield fid.read(frameSize)\n",
    "            pos = pos + frameSize\n",
    "\n",
    "for frame in getFrame('SomeFilePath'):\n",
    "    print(frame)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "对于以上代码，我不过多讲解。大家对于其中有不明白的地方，可以自行查找帮助手册或者百度。如果还有不明白的地方，可以留言。其中大家可以重点理解with用法，这是在文件读取时常用并且非常好用的一个语法。"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
